NetDispatcher : A TCP Connection Router

NetDispatcher is a software router of TCP connections that supports load sharing across multiple TCP servers. It consists of the Executor, an operating system kernel extension that supports fast IP packet forwarding, and a user level Manager process that controls it. The Manager implements a novel dynamic load-sharing algorithm for allocation of TCP connections among servers according to their real-time load and responsiveness. This algorithm produces weights that are used by the Executor to quickly select a server for each new connection request. This allocation method was shown to be highly e cient in real tests, for large Internet sites serving millions of TCP connections per day. The Executor forwards client TCP packets to the servers without performing any TCP/IP header translations. Outgoing server-to-client packets are not handled by NetDispatcher and can follow a separate network route to the clients. Depending on the workload tra c, the performance bene t of this half-connection method can be signi cant. Prototypes of NetDispatcher were used to scale up several large and high-load Internet sites.